Conventionally, a stereo image processor is known, which calculates partial displacement between images based on two images (a base image and a comparative image) obtained when a stereo camera is used to take images of the same object, and measures a distance to the object based on the partial displacement between the images. The stereo image processor is under study for application to devices such as one that measures a distance to a vehicle in front based on images obtained by using an on-board camera to take the images of the vehicle or one that measures a distance to a part of the driver's face, such as an eye and a nose, based on images obtained by using an inboard camera to take the images of the face and estimates the direction of the driver's face. However, the smaller size of recent cameras, such as on-board and inboard cameras, have led to a smaller camera spacing and resultant smaller partial displacement between stereo images. Consequently, there is a need for highly accurate disparity calculating functions for a stereo image processor.
Conventionally, a stereo matching method (which is for disparity calculation for stereo image processing) such as a sum of absolute difference (SAD) method and a phase only correlation method (POC) is used in such a stereo image processor.
In a SAD method, partial images are cut off from a base image and a comparative image respectively by using a rectangular window, and the total sum of absolute values of difference between luminance values of the partial images is calculated. Characteristics values here, such as a SAD value, indicate a level of difference in luminance of images. The position of the rectangular window of the comparative image is then shifted on a per pixel basis in the baseline direction to find a alignment at which the SAD value is minimized, which is defined as “pixel level disparity (i.e. partial displacement)”. Thereafter, three SAD values around the minimum value (i.e. the minimum, the second minimum, and the third minimum of SAD values) are used to calculate “sub-pixel level disparity (i.e. partial displacement)” by isometric linear fitting.
Such a SAD method has traditionally been used and characterized by relatively high analytical resolution with less computation. However, the SAD method suffers from low accuracy of sub-pixel level disparity calculation; the SAD method can determine disparity (i.e. partial displacement between images) only on the order of ¼ to 1/16 pixel accuracy and is difficult to satisfy the need for highly accurate disparity calculating functions.
Recently, therefore, the POC method draws attention for its high accuracy in disparity calculation. In the POC method, partial images are cut off from a base image and a comparative image respectively by using a window function for reducing an effect from harmonics occurring when a Hanning window or the like is used to cut off an image, and a 2D Fourier transformation is performed on the partial images. The 2 pieces of Fourier image data are combined and the amplitude component is normalized. A 2D inverse Fourier transformation is then performed on the data to determine a phase-limited correlation coefficient. The amount of partial displacement between images is then determined based on a correlated peak.
Such a POC method, which is referred to as a 2D POC method, has an advantage of very high accuracy in disparity calculation. However, the 2D POC method requires a large amount of computation in disparity calculation, and it is difficult to compute in a short time. In addition, the 2D POC method is inferior to the SAD method in terms of analytical resolution, which is the quantity on a screen at which isolated objects can be distinguished and the distance can be measured.
Recently, a 1D POC method is proposed (see Patent Literature 1), which requires less computation than the 2D POC method. In the 1D POC method, partial images are cut off from a base image and a comparative image respectively by using a Hanning window, and a 1D Fourier transformation is performed on the partial images. The 2 pieces of Fourier image data are combined and the amplitude component is normalized. A 1D inverse Fourier transformation is then performed on the data to determine a phase-limited correlation coefficient. In other words, the 1D Fourier transformation is performed instead of the 2D Fourier transformation to reduce computation.
However, even though computation has been somewhat reduced in the conventional 1D POC method, the reduction is still insufficient and computation required to calculate disparity is still much greater (in comparison with the SAD method); therefore, it is not easy to compute in a short time. In addition, the 1D POC method is significantly inferior to the SAD method in terms of analytical resolution, which is the quantity on a screen at which isolated objects can be distinguished and the distance can be measured.